Papers by Varun R. Embar
Eliciting In-context Retrieval and Reasoning for Long-context Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks like LOFT often overestimate LCLM performance by providing overly simplified contexts. |
| Approach: | They propose to use retrieval-attention-probing to filter and de-noise long contexts during decoding and joint retrieval head training alongside the generation head to improve LCLM performance. |
| Outcome: | The proposed approach outperforms RAG and GPT-4-Turbo on most tasks despite being a much smaller model. |